Computational Beauty Analysis

Our project was a continuation of the recent research published by Yael Eisenthal, Gideon Dror, and Eytan Ruppin entitled: Facial Attractiveness: Beauty and the Machine [1]. The authors used a variety of machine learning techniques to predict facial attractiveness ratings from photographs. They had two data sets, each consisting of ninety-two photographs of young, Caucasian women. All images in the first data set were taken by the same photographer under the same lighting conditions and with the same orientation. These high-resolution photographs were of Americans with neutral facial expressions who were wearing no glasses or jewelry. The second data set was of somewhat lower quality. The women in these photographs were Israelis, some of whom were wearing jewelry or smiling with closed lips. Eisenthal, Dror, and Ruppin used two distinct representations of the women’s faces: the feature representation and the pixel representation. The feature representation consisted of distance measurements between feature landmarks on the face and ratios between these distances. Examples of feature landmarks include the centers of the pupils, the corners of the mouth, and the endpoints of the eyebrows. All distances were normalized by the distance between pupils. Additionally, average hair color, skin color and skin smoothness values were included as part of the feature representation. The image representation was simply the original photograph converted to grayscale and concatenated by columns to become a vector. The authors of [1] used both classification and regression techniques. For classification, they retained only the photographs with average attractiveness ratings in the top or bottom quartile. They labeled these photos as “attractive” or “unattractive” accordingly. Their best classification results are summarized in Fig. 3. They attempted several kernels but found that a linear kernel worked best for SVM. They also tried K-nearest neighbors (KNN). Besides classification, [1] used the regression version of SVM to predict attractiveness ratings. They were able to achieve a 0.65 correlation on a test set by combining the predictions from the feature and pixel representations, but the feature representation alone performed nearly as well with a 0.6 correlation. A very promising result was the shape of the learning curve. (See Fig. 4) By extrapolation, it appears that significantly higher correlations can be achieved by using a moderately larger data set. This expectation was a big motivator for this project, which leads to our objective: to repeat the work of Eisenthal, Dror, and Ruppin, but using a larger training set.